1. Field of the Invention
This invention relates to the field of compilers, and more particularly, to a parser for use in a compiler that utilizes a dynamically extensible approach to parsing textual input.
2. Description of Related Art
—The analysis and parsing of textual information is a well-developed field of study, falling primarily within what is commonly referred to as ‘compiler theory’. At its most basic, a compiler requires three components, a lexical analyzer which breaks the text stream up into known tokens, a parser which interprets streams of tokens according to a language definition specified via a meta-language such as Backus-Naur Form (BNF), and a code generator/interpreter. The creation of compilers is conventionally a lengthy and off-line process, although certain industry standard tools exist to facilitate this process such as LEX and YACC from the Unix world. There are a large number of textbooks available on the theory of predictive parsers and any person skilled in this art would have basic familiarity with this body of theory.
Parsers come in two basic forms, “top-down” and “bottom-up”. Top-down parsers build the parse tree from the top (root) to the bottom (leaves), bottom-up parsers build the tree from the leaves to the root. For our purposes, we will consider only the top-down parsing strategy known as a predictive parser since this most easily lends itself to a table driven (rather than code driven) approach and is thus the natural choice for any attempt to create a configurable and adaptive parser. In general, predictive parsers can handle a set of possible grammars referred to as LL(1) which is a subset of those potentially handled by LR parsers (LL(1) stands for ‘Left-to-right, using Leftmost derivations, using at most 1 token look-ahead’). Another reason that a top-down algorithm is preferred is the ease of specifying these parsers directly in BNF form, which makes them easy to understand by most programmers. Compiler generators such as LEX and YACC generally use a far more complex specification methods including generation of C code which must then be compiled, and thus is not adaptive or dynamic. For this reason, bottom-up table driven techniques such as LR parsing (as used by YACC) are not considered suitable.
What is needed is a process that can rapidly (i.e., within seconds) generate a complete compiler from scratch and then apply that compiler in an adaptive manner to new input, the ultimate goal being the creation of an adaptive compiler, i.e., one that can alter itself in response to new input patterns in order to ‘learn’ to parse new patterns appearing in the input and to perform useful work as a result without the need to add any new compiled code. This adaptive behavior is further described in Appendix 1 with respect to a lexical analyzer (referred to in the claims as the “claimed lexical analyzer”). The present invention provides a method for achieving the same rapid, flexible, and extensible generation in the corresponding parser.